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ABSTRACT 


Creativity is a dynamic process which generates ideas that 
are both novel and of value. However there is little under- 
standing in what drives creativity in students and how to 
help teachers or education experts to detect creative think- 
ing. This paper begins to address this gap by providing 
a platform and experiments for studying how creative out- 
comes can ensue over time. The platform is an open ended 
environment for creating geometrical shapes that supports 
exploration and trial and error. We show that participants 
exhibit diversity in their usage patterns in the system, and 
in particular, some exhibiting ’creative leaps’ in which they 
move from creating a sequence of shapes in one category 
to another, new category. We designed a visualization tool 
that aids understanding in detecting these aspects in stu- 
dents’ work. We provide a basic computational model that 
is able to predict whether a student will create a new shape 
at a given point in time. The impact of this work is in be- 
ginning to provide tools for promoting creativity in students 
and directing their interactions in a way that facilitates the 
creative process. 


Keywords 
Creativity, Divergent Thinking, Geometry, Geogebra 


1. INTRODUCTION 


There is multiple evidence that exhibiting creativity in the 
classroom is linked to positive learning gains, and using ed- 
ucational technology to bring about creativity is an active 
area of research [19, 8]. To date, however, such technologies 
have relied on human teachers to detect and to promote cre- 
ative behavior. While research on adaptive technologies for 
education have flourished, there are few studies on automat- 
ically detecting creativity from students’ interactions. 


This paper begins to address this gap by providing a plat- 
form for studying and computationally modeling creative 
tasks. The task requires participants to create geometric 
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shapes and explore multiple solutions in a domain that is 
simple to define and explain, while still providing a rich 
space of possible solutions. 


The task was chosen so that there is no single correct answer, 
and the goodness of a solution is measured by the number 
and quality of the different answers. This task supports a 
process of exploration and discovery. There are many pos- 
sible strategies for solving the tasks, some requiring more 
skills than others. 


We study people’s search trajectories in the space of possible 
solutions, showing that people exhibit creative leaps [9], al- 
ternating between clusters of solutions and exploration. We 
adapt a model by Leikin [10] for measuring creative out- 
comes in users’ interactions, based on defining their work 
in terms of flexibility, originality and fluency. We provide a 
visualization tool that decomposes a user’s interaction se- 
quence in the system to separate sequences of solutions. 
When a creative leap occurs, the solutions in the inferred 
sequence belong to the same class. 


We show that people’s creative outcomes in the system varies 
widely, in a way that depends on the creative leaps that 
are exhibited by the participants. We built a computational 
model that attempts to predict whether or not a given shape 
is new for a given participant. We define several sets of fea- 
tures that include statistics about shapes created as well as 
GUI operations used to create the shapes. The best perfor- 
mance was achieved by a random forest classifier that was 
based on both features. 


These results can potentially inform the design of algorithms 
for detecting and promoting creative outcomes. 


The remainder of this paper is organized as follows. In the 
next section we provide a general description of the creativ- 
ity testbed for creating geometric shapes. We then describe 
a tool for visualizing the shapes created by individual users 
over time, and how to cluster these shapes in a way that can 
detect creative leaps. Finally we provide a computational 
model for detecting new shapes in a user’s interactions, and 
discuss ways to extend this model to detect creative out- 
comes. 


2. RELATED WORK 


Previous research showed that creativity is linked to positive 
learning gains, and using educational technology to bring 
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about creativity is an active area of research [19]. While 
these programming environment support interactions that 
can lead to creative thinking, they rely completely on human 
teachers to detect and to promote this behavior. 


We focus on the use of technology to promote divergent 
thinking, which is a type of creative ability that generates 
multiple answers to problems [18, 6]. Guilford [6] defines 
divergent thinking as generation of multiple answers to a 
problem. Torrance[17] defined fluency, flexibility, novelty, 
and elaboration as parameters that describe divergent think- 
ing. Fluency is the ability to create large number of ideas for 
a problem that are useful. Flexibility is the ability to change 
direction, thinking strategies and point of views. Original- 
ity is the ability to generate novel and unconventional ideas. 
Unconventional ideas defined as statistically infrequent. 


There are several works focusing on the use of technology 
to promote creative outcomes in students. Multi Solution 
Tasks (MST) is mathematics open ended problems with mul- 
tiple correct answers that can reached in different ways [12]. 
MST improves the participant’s understanding and the con- 
nection between his knowledge domains, skills and strategies 
[4]. Levav-Waynberg & Leikin [12] found that MSTs raised 
the connection between knowledge domains in geometry, and 
improve the fluency and flexibility of the participants. Ge- 
ometry plays a major role in Math teaching. It includes 
visual, abstract and logical skills. Geometry created op- 
portunities for investigation, generalization, deduction and 
gives autonomy to the learner to explore mathematics with 
his personalized preferences [12, 3]. 


Sophocleous & Pitta-Pantazi [16] found that using software 
environment for geometry enhanced the creative abilities of 
students by facilitating them to provide more, different and 
unique solutions. 


Noy et al. [15] demonstrated the role of creative leaps in two 
dimensional geometry. They show that human players ex- 
hibit two types of exploration: ’scavenging’, where similar 
shapes are accumulated, and ’creative leaps’, where players 
shift to a new region in the shape space after a prolonged 
search. They show that the network of shapes created by 
human participants is different from the class of networks 
created by applying a simple random-walk algorithm. We 
extend their work in two ways. First by providing a com- 
putational model to detect new shapes; second in extending 
the notion of creative leaps to a framework that allows a 
richer set of actions to be created. In subsequent work they 
studied creative exploration using a scale invariant model 
that considers relative changes in signals [7]. 


There are few studies on automatically detecting creativity 
from students’ interactions. An exception is Manske and 
Hoppe [14], who have used supervised learning methods to 
detect creativity in programming assignments that require 
mathematical skills. They combined low level features (e.g, 
code snippets) with higher level features (e.g., the use of 
recursion, number of lines of code) to train numerical pre- 
dictors and predicted a creativity score for new solutions. 
Chuang et al.{2] used fuzzy logic to detect student’s creativ- 
ity measures in a gaming environment. Loveless et al.[13] 
studied the use of technology to promote aspects of cre- 
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Figure 1: Open Creativity App 


ative thinking for student teachers, including the develop- 
ment of new ideas, modifying and evaluating the original- 
ity and value of work as it develops. These works relied 
on manual approaches for detecting creativity and did not 
study how to visualize these creative outcomes. 


3. THE GEO CREATIVITY TESTBED 


We designed an activity (built as a GeoGebra app) in which 
participants create geometric shapes by manipulating the 
shaded area that intersects two rectangles (See Figure 1). 


Participants can employ geometric transformations on each 
rectangle according to several possible actions supporting 
by the testbed GUI: Translation (shifting a rectangle along 
the x or y axis), rotation (re-positioning the rectangle by 
changing one or more of its angles), re-sizing (increasing or 
decreasing the size of both of the rectangles). Creating dif- 
ferent shapes requires to master different skills. It is easier to 
create polygons of varying number of sides by rotating the 
rectangles, but other types of shapes requires more steps, 
using more actions or have a precise positioning of the rect- 
angles in designated angles. 


To help with the positioning, participants can optionally 
choose to display the angels formed at the vertices, the 
length of the intersection shape sides, or position the two 
rectangles perpendicularly one to each other. At any point 
in time, participants can choose to submit their shape to a 
gallery. When the shape submitted to the gallery, the inter- 
face doesn’t change and the participates continue their work 
from the same point. 


The GUI design supports several key factors that have been 
shown to facilitate creative outcomes in students. First, it 
provides an open ended task in which there is no single cor- 
rect answer, and the goodness of a solution is measured by 
the number and quality of the different answers [10]. 


Second, the task supports a process of exploration and dis- 
covery. Participants can manipulate two rectangles to create 
new shapes. Trial and error is a key part of the exploration 
process [5]. 


Third, understanding the task does not rely on complex or 
unique tool set. Participants with little knowledge in ge- 
ometry can use the task and think about novel and useful 
categories. Participants with more knowledge and experi- 
ence will have a better potential to think about new and 
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Figure 2: Categories Distribution over Participants 


unconventional categories [1]. 


Fourth, there are many strategies to create shapes from the 
same category. Some requiring more skills or using more 
tools than others. For example, squares can be created by 
rotating and positioning the two rectangles, and also by us- 
ing “perpendicular rectangles” option in the GUI menu [1]. 


This task is recommended activity by content developers for 
K4-K5 students that learn geometry. The idea is for students 
to learn to generate geometric shapes in a novel way (inter- 
secting the two rectangles). The number of possible shapes 
that students can create with the system is not bounded. 


3.1 Procedure 

We recruited 183 Participants (87 undergraduate students 
and 96 Mturk workers with varying educational background 
- high school, Bachelor and master graduates). All partic- 
ipants needed to pass a tutorial and comprehension quiz 
about the study in order to participate. 


All participants were requested to ”create as many different 
polygons types as possible by intersecting two rectangles.” 
Students performed the task as part of an extracurricular 
activity and were not monetarily compensated. Participants 
in Mturk received monetary compensation as follows: 30 
cents as a show up fee and 10 cents for every type of shape 
that they’ve created. 


The task was limited by 10 minutes, and on average partic- 
ipants spent 4.5 minutes on the task. 


Participants created between one and ten different shape 
categories (e.g., Polygon, square, etc.) and 1445 shapes in 
total. Figure 2 shows the frequency of the shape categories 
submitted by users to their portfolio. The x axis denotes the 
shape category; the blue bars display the number of shapes 
that was created for each category (values in the left vertical 
axis). The red line shows the percentage of participates that 
created shapes of the given category (values in the right 
vertical axis. 


As shown by the figure, the most popular shape categories 
were Pentagon, Hexagons and other” Quadrilaterals (quadri- 
laterals which were not rhombus, square or rectangles). The 
least popular shape categories were septagon, square and 
rectangles. There was a general correspondence between 
the number of times a shape was created and the number 
of users who created the shape. However, some shapes cat- 
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Figure 3: Visualization tool showing timeline of 
shapes (top), and shape sequences (bottom) for user 
ID 752 


egories, namely Rhombus, square and rectangle, were more 
popular. For example, the Rhombus is the fourth most pop- 
ular shape category, yet it is the second most popular shape 
among the users. We will show later that these shapes played 
a special part in people’s creative process. 


3.2 Visualization Tool 

We designed a visualization tool for studying how individ- 
ual participants create shapes over time. Fig 3 shows the 
main interactive panel in the visualization system. The main 
panel shows the shapes created by an individual user (ID 
752). The « axis represents time (in seconds) from com- 
mencement of the interaction, while the y axis represents 
the length of time (in seconds) it took to create shapes. For 
example, the coordinate (30,4) shows the first shape cre- 
ated by the user (Rhombus) at time 30 and took 4 seconds 
to create. As shown in the figure, participant ID 752 cre- 
ated 11 shapes over a time span of 230 seconds. We can see 
that the participant exhibited high variance in the creation 
time of the shapes. For ease of analysis, there is a way to 
group shapes into temporal sequences according to the fol- 
lowing criteria: First, a sequence has to include at least n 
contiguous shapes. Second, the probability that the next 
action belongs to the same sequence is greater than a desig- 
nated threshold T. The threshold can be set in a way that 
maximizes the number of shapes in the sequence. A shape 
i commences a new sequence if P( aise ) > T where A; is 
the length of time spent to create shape i, and o are the 
average time and standard deviation for creating shapes by 
the participant. In this way we consider the extent to which 
the creation time of each shape agrees with the individual 
participant (assuming a normal distribution over creation 
time). 


Figure 3 shows four shape sequences that were created by 
the user, inferred by the criteria described above. We can 
see that sequence II and IV are relatively short and include 
shapes that share a common category (that of trapezoids or 
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hexagons, respectively). In contrast, sequence I and III are 
longer and include shapes of different categories. We next 
show how these sequences yield insights into creative aspects 
of the shape creation processes. 


3.3. Measuring Creativity 

We measure the creativity of a participant following Tor- 
rance [17] who defined dimensions of flexibility, fluency and 
originality to describe creative solutions to problems: Flu- 
ency is the number of solutions that was proposed by the 
participant. Flexibility is the number of strategies that the 
participant applies for the solution. Originality is statisti- 
cally infrequent ideas that were produced by students rela- 
tive to their classes or groups. 


We calculate the score by the formula suggested by Leikin 
[11], adapted to the geometry app. The flexibility of a shape 
7 is 10 if the shape is new for this participant, or 1 otherwise. 
The originality of a shape 7 is 10 if the shape was chosen by 
fewer than 15% of participants (septagons), 1 if the shape 
was chosen by between 15% and 40% of participants (rect- 
angles and squares), and 0.1 if the shape was chosen by more 
than 40% of participants. 


The fluency of a shape is always 1. Let n be the number 
of shapes created by the participant (also the participant’s 
fluency score). The creativity score of a participant is com- 
puted as )0"_, FX;-OR; where FX; and OR, are the flex- 
ibility and originality of shape i. 


4. CREATIVE LEAPS 


Participants exhibited a diverse range of creativity scores in 
their work. The average score was 80 with a standard devia- 
tion of 54. To explain differences between students, we need 
to analyze the dynamics of how shapes were created. We 
will distinguish between two types of shape creation, those 
representing exploration in the space of possible concepts, 
and those representing exploitation of one of the concepts 
that is used to create shapes. 


Koestler [9] describes a creative leap as the moment where 
a new dimension of possibilities appears. The creative leap 
signifies a point in the search space in which the learner 
discovers a new class of solutions and begins to exploits this 
space by creating shapes. 


In sequence II and IV of participant 752 (shown in Fig- 
ure 3) the participant exploits the concept of trapezoid and 
of hexagons, completing two shapes in each category. Both 
of these sequences represent creative leaps. In contrast, se- 
quence I and III for this user represents exploratory behavior 
in which the participant does not converge on a shape cate- 
gory. In particular, sequence IV commenced 81 seconds after 
the last shape, suggesting a lengthy process of exploration 
leading to the next shape category. 


We use the concept of creative leaps to distinguish between 
different types of participants, as determined by their inter- 
action in the system. 


Another example, participate 651 holds an MA degree, ex- 
hibited a creativity score of 128, with fluency of 36, flexibility 
of 8, and originality of 1 (rectangle and square). Participant 
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Figure 4: Shape sequences for low scoring partici- 
pant ID 655 (left) and high scoring participant ID 
651 (right) 
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Figure 5: New and old shapes 


655 holds a high school diploma, with a creativity score of 
61, with fluency of 5, flexibility of 4 and originality of 1 
(rectangle and square). 


Figure 4 shows the shape sequences for these two partici- 
pants. As shown by the figure the users exhibited drastically 
different interaction styles with the system. The low scor- 
ing participant (user ID 655) created only five shapes, which 
can be described by 4 sequences. Only sequence I for this 
user consisted of shapes of a similar category. In contrast, 
the high scoring participant (user ID 651) created 36 shapes, 
which can be described by 10 sequences (for brevity we only 
show the first four). Five of these sequences included shapes 
with particular categories. 


5. COMPUTATIONAL MODEL 


Using the creation of a new shape as a proxy for creativity, 
we build a computational model that attempts to predict 
whether or not a given shape is new for this participant. 
This is a first step for detecting creativity for this partici- 
pant. To this end, we extract a number of features and use 
logistic regression and a random forest classifier to predict 
the binary outcome of whether or not a given shape is new 
for the user. 


Figure 5 shows the layout of new shapes (green) and old 
shapes (blue) for users across time. It demonstrates that 
while the proportion of new shapes being generated certainly 
does decrease as the user continues to interact on the plat- 
form, there are a significant number of shapes that are cre- 
ated ‘later’ on in the user’s interaction session. These shapes 
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Figure 6: Plots showing the raw shape coordinates 
and the extracted actions from the database. Top 
is a resize action where both input rectangles are 
affected. Bottom is a rotation action to the second 
rectangle 


are new to the user and can signify the commencement of a 
creative leap. 


5.1 Feature Extraction 

The raw data represents individual coordinates of the two 
input rectangles where the user can rotate, translate and re- 
size the two rectangles. We extract changes to these coordi- 
nates to represent the 5 actions that the user can perform on 
the system. Figure 6 shows the primary actions that a user 
can perform. The resize action affects both shapes simulta- 
neously whereas the user can perform rotate and translate 
actions to each shape individually. 


As a number of actions are preformed before a user submits 
a shape, we include the history of actions in the feature ma- 
trix considering a maximum of 15 actions before the shape 
was submitted; 95% of shapes were submitted with less than 
15 steps. We aim to infer whether including the history of 
actions provides additional information to a model predict- 
ing the creation of a new shape. In other words, our goal is 
to determine if combinations of actions are predictive of a 
new shape and thus indicative of certain creative insights. 


5.2. Method 


Given four feature sets, we use cross-validated logistic regres- 
sion (LR) and random forest (RF) classification to predict 
the binary classification problem of whether or not the shape 
is new for the user. Other classification techniques were ex- 
plored including support vector machines, boosted decision 
trees and naive Bayes classifiers but LR and RF were cho- 
sen for simplicity and the ease of interpretation into the 
parameter coefficients (or feature importance in the RF). 
Moreover, LR allows the application of an L1 sparsity reg- 
ularizing parameter to induce sparsity in the feature space 
(thereby assisting inference). The available features are: 


Table 1: Different combinations of available features 
that the four feature sets used. 


Feature Set] Shape History | Action History | Aggregated Action 
History 


Feature set 1 
Feature set 2 Y Y 


Feature set 3 Me 


KK KK 


Feature set 4 ¥. 


1. Shape History. Counts of previously submitted shapes 
from the user for each of the 10 shape categories. 


2. Action History. A one-hot encoded representation 
of actions in 15 previous steps leading into the shape 
creation. Following the actions shown in Figure 6 there 
were 3 possible actions (translate, rotate or resize) at 
each step. We further include a ’control’ action that 
represents a recorded step but no action. This case 
might occur when the user does not interact with the 
shapes but rather interacts with a different UI element 
or possibly performs an administrative action such as 
submitting the shape. This creates 60 features in total 
for each instance. 


3. Aggregated Action History. The sum of the Ac- 
tion History features across the 15 steps. This re- 
sults in 4 additional features per instance describing 
the number of times each action was performed. For 
example, a vector (2,3, 5,5) corresponds to 2 translate, 
3 rotate, 5 resize and 5 control actions (in any order). 


The three sets of available features were aggregated in dif- 
ferent combinations to provide four feature sets that were 
used for evaluation. Table 1 summarizes the different com- 
binations that were used. 


5.3 Results 


Our goal was to determine if certain sequences 

and/or combinations of actions are predictive of the user 
generating a new shape. Table 2 summarizes these results. 
We note that the Feature Set 3 provides a slight improve- 
ment over the baseline of Feature Set 1 for both LR and RF. 
Feature Set 4 with RF shows the greatest accuracy with a 
2% increase over the baseline feature set. The results are 
reported from a 10 fold cross-validation but the predictive 
increase is not significant across the folds. The results sug- 
gest that the inclusion of the action data does assist slightly 
with the predictive performance of the model but we note 
the result is not conclusive. However, analyzing the feature 
weights of the LR for Feature Set 3 and the RF for Feature 
Set 4 is illuminating. 


It is interesting to note that the RF model with the tem- 
poral features outperforms the LR on these same feature 
sets. Again, although the results are not significant they 
do suggest there is a more complex interaction of the action 
history of the user that might be predictive of the new shape 
creation. Further investigation into how sequences of ac- 
tions might be indicative of the creation of a new shape (and 
thereby indicative of a creative leap) is needed to answer this 
question definitively. 
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Table 2: Table showing the results from the pre- 
diction task of predicting a new shape for a given 
user. 


Accuracy 
Feature Set LR RF 


Trivial all 0 prediction 


Feature set 1 
Feature set 2 
Feature set 3 
Feature set 4 


6. CONCLUSION AND FUTURE WORK 


We presented an approach for studying creativity using a 


web based tool in which participants created geometric shapes. 


This task supports a process of exploration and discovery, 
allowing people to exhibit creative leaps in which they tran- 
sition between different areas of the search space of possible 
solutions. We adapted a model by Leikin for measuring 
creative outcomes in users’ interactions. We collected data 
from multiple people interacting with the system showing 
that they vary widely in terms of the creativity they ex- 
hibit. We built a visualization tool that decomposes a user’s 
interaction sequence in the system to separate sequences of 
solutions. We built a computational model that attempts 
to predict whether or not a given shape is new for a given 
participant. We define several sets of features that include 
the number and categories of shapes that were created by 
the user, as well as basic actions performed by the users in 
the system for a window of activity. The best performance 
was achieved by a random forest classifier that was based 
on both features. In future work we intend to extend the 
computational model to detecting creative outcomes in new 
types of domains. 
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